Classifiers: A Theoretical and Empirical Study

Th is paper describes how a compet i t i ve tree learning a lgo r i t hm can be derived f r o m f i rst pr inciples. The a lgo r i t hm approximates the Bayesian decision theoret ic so lut ion to the learning task. Compara t ive exper iments w i t h the a lgo r i t hm and the several matu re AI and stat is t ica l famil ies of tree learning a lgor i thms cur rent ly in use show the der ived Bayesian a l g o r i t h m is consistent ly as good or bet ter , a l though sometimes at computa t iona l cost. Using the same strategy, we can design a lgor i thms for many other supervised and mode l learning tasks given jus t a probabi l is t ic representation for the k i n d of knowledge to be learned. As an i l l us t ra t ion , a second learning a lgo r i t hm is der ived for learn ing Bayesian networks f r o m data. Impl icat ions to incrementa l learning and the use of mu l t i p le models are also discussed.